1 METHOD AND APPARATUS FOR VALIDATING 

2 CROSS-ARCHITECTURE ISA EMULATION 

3 Technical Field 

4 The technical field is emulation of computer instruction sets. 

5 Background 

6 Cross-architecture emulation is needed when running native applications on a 

7 target platform, or computer, that uses an instruction set architecture (ISA) different from 

8 the ISA for which the native applications were initially intended. The developer of a 

9 cross-architecture emulation product wants to know that the emulator correctly translates 

10 or interprets the native applications. Various schemes exist to verify cross-architecture 

1 1 emulation. However, these schemes cannot easily, if at all, comprehensively test the 

1 2 emulation process. 

1 3 To improve the emulation process, a binary translation product that runs on the 

1 4 target platform may be used to emulate native instructions running on a native, or legacy, 

15 platform. Binary translation automatically translates binary code from the native 

16 instruction set to binary code for the target platform without the need for high-level 

17 source code. As with any emulation product, a developer of the binary translation 

1 8 product may desire to verify the accuracy of emulation from the native ISA to an ISA 

1 9 operating on a target platform. 

20 One conventional approach to verifying cross-architecture emulation is to send 

21 as many native applications as possible through the binary translation product, or 

22 emulator, and then verify that the outputs produced by the binary translation product are 

23 identical to corresponding outputs produced by the native applications running on the 

24 native platform. This conventional approach has several disadvantages. First, the binary 

25 translation product designer cannot know whether testing is complete because the native 

26 applications are usually compiler-generated and the machine code generated by the binary 

27 translation product only includes a subset of the instruction set architecture. Machine 

28 instructions that are not in this subset will never be generated in the compiler so that 

29 some binary instructions are never tested through the emulation process. Second, running 

30 an application through the binary translation product on the target platform and running 

3 1 the same application through the native platform is cumbersome. A separate process may 

32 then be needed to verify the results. Third, when an error is detected, pinpointing the 
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1 exact machine instruction that caused the error may be difficult or impossible. In 

2 addition, replication of the emulation error may be impossible to achieve. Fourth, 

3 execution of some applications is time- or system-sensitive such that the result of an 

4 execution may not be reproducible, v^hich adds to the difficulty ov verifying binary 

5 emulation. 

6 Summary 

7 A method and an apparatus allows complete and efficient verification of cross- 

8 architecture ISA emulation. A random verification framework runs concurrently on two 

9 different computer architectures. The framework operates without regard to existing 

10 native applications and relies instead on binary instructions in a native ISA. The 

11 framework is able to determine emulation errors at a machine instruction level. The 

12 emulation errors are then easily identifiable and reproducible. 

13 A random code generator generates one or more sequences of native machine 

14 instructions and corresponding initial machine states in a pseudo-random fashion. The 

15 native instructions are generated from an entire set of the native ISA. The instructions 

16 and the state information are provided to initialize a native computer architecture. The 

1 7 same instructions and state information are provided using standard machine-to-machine 

18 languages, such as TCP/IP, for example, to a target computer architecture. The target 

19 computer architecture may be embodied as an actual hardware device, or may be a 

20 simulation of a yet-to-be-built computer architecture. The target computer architecture 

2 1 includes a binary emulator that translates the native instructions into binary instructions 

22 executable on the target computer architecture. The final states of the native and the 

23 target computer architectures are gathered, and a verification engine compares the results. 

24 Any differences may indicate an emulation error or failure. 

25 The random verification framework may be run continuously to test emulation of 

26 the complete set of instructions from the native ISA. Even the least-used machine 

27 instructions are tested by the framework. Further, the fi-amework automates the 

28 emulation verification process. Inter-machine communications allow the native and the 

29 target computer architectures to process the same machine instructions fi-om the same 

30 initial states. Any inconsistencies in the final produced states indicate the emulation 
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error. The framework can then easily pinpoint the exact machine instruction, register 




2 


number and input machine state that caused the emulation error, thereby significantly 




3 


reducing the amount of time required for debugging. Finally, the emulation errors 




4 


detected using this fi-amework are easily reproducible. 
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Description of the Drawings 
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The detailed description will refer to the following drawings, in which like 




7 


numerals refer to like objects, and in which: 
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Figure 1 is a block diagram of an example of an apparatus for validating cross- 
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architecture ISA emulation; and 




10 


Figure 2 is a flowchart illustrating an operation of the apparatus of Figure 1. 
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Detailed Description 


m 


12 


A method and an apparatus allow complete and efficient verification of cross- 


I tss 


13 


architecture ISA emulation. A random verification framework runs concurrently on two 




14 


different computer architectures. The framework is able to determine emulation errors 


;n 


15 


at a machine instruction level. The emulation errors are then easily identifiable and 




16 


reproducible. 




17 


Figure 1 is a block diagram of an embodiment of an apparatus for validating 


y 


18 


cross-architecture instruction set architecture (ISA) emulation. A framework 1 0 includes 


3 


19 


a random code generator (RCG) 20 that generates native machine instruction level code 


=1 


20 


and an initial machine state. The machine instruction level code (i.e., sequences of binary 




21 


instructions) and initial machine state are provided to a native ISA platform 11, which 




22 


includes a first, or X, execution engine 30. The X execution engine 30 is capable of 




23 


executing the instruction level code without emulation or translation. 




24 


The RCG 20 may include a probability file that is used to pseudo-randomly 




25 


generate the machine instruction level code. The RCG 20 also provides the machine 




26 


instruction level code and the initial machine state to a non-native ISA or target platform 




27 


12, which includes a second, or Y, execution engine 40. To execute the machine 




28 


instruction level code, the Y execution engine 40 emulates or translates the machine 




29 


instruction level code using an emulator 45. 



HP 10001205-1 




4 

1 In an alternative embodiment, the emulator 45 may operate on a target platform 

2 simulator (not shown), which in tum operates on the native platform 1 1 . 

3 The X execution engine 30 and the Y execution engine 40 may execute the same 

4 machine instmction level code concurrently. The execution engines may perform 

5 information transfer using any standard machine-to-machine protocol, such as TCPAP, 

6 for example. 

7 The X execution engine 30 and the Y execution engine 40 each will provide a 

8 final machine state. The final machine states are sent to a verification engine 50. The 

9 verification engine 50 compares the final machine states to determine any differences. 

10 Such differences may indicate an error in emulation of the native machine instruction 

1 1 level code. In particular, the verification engine 50 is able to pinpoint the exact machine 

1 2 instruction, register number and input machine state that caused the emulation error. The 

13 exact source of the emulation error may be particularly easy to locate because the 

1 4 instruction sequence that generated such an emulation error is typically a short sequence 

1 5 of binary instructions. Hence, a review of information saved when an emulation failure 

16 occurs allows the test designer to quickly pinpoint the exact cause of the emulation 

1 7 failure. 

18 In an example, if a binary instruction to be emulated comprised ADD Rl and R2, 

19 R3, the verification engine 50 would determine if the result written to the register R3 

20 using the emulator 45 and the Y execution engine 40 was the same as that written to the 

21 register R3 using the X execution engine 30. As long as the final state produced by the 

22 X execution engine 30 is the same as the final state produced by the Y execution engine 

23 40, the emulation of the binary instruction with a given specific initial machine state was 

24 correct. 

25 The fi-amework 10 relies on pseudo-random generation of machine level 

26 instructions to comprehensively test the correct emulation of native applications on a 

27 second computer architecture. The machine level instructions may be generated 

28 according to many different random generation schemes. In an embodiment, each 

29 machine level instruction is assigned a probability. In particular, a specific instruction 

30 to be generated is controlled by an input probability file, which is a list of pre-defined 
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1 machine instructions, each with an associated probability. The machine instructions are 

2 arranged in a hierarchy form and divided into segments based on a function of the 

3 instruction. A first such segmentation may include floating instructions and CPU 

4 instructions. Continuing, the CPU instructions may be further segmented according to 

5 arithmetic/logic, immediate, memory, system, memory management, and branch 

6 instructions, and other CPU instructions. The arithmetic/logic instructions may be 

7 segmented into ADD, SUB, AND, OR, and XOR instructions, and other arithmetic/logic 

8 instructions, for example. Each hierarchical instruction is assigned a probability such that 

9 a cumulative probability along any hierarchical path equals 1.0. 

10 The instruction sequence generation process is pseudo-random because any 

1 1 random code generator must operate according to a pre-determined algorithm. Further 

12 limitations also lead to a loss of true randomness. For example, certain memory regions 

13 may be inaccessible to the random code generator. 

14 An example probability file that controls the pseudo random instruction 

15 generation follows. In the example, expressions such as pr_xxx refer to a probability 

16 value for binary instruction xxx. 

17 The instructions are arranged in hierarchical segments. 

18 pr_cpu =1.00 

19 pr_fp =0.00 
20 

21 pr_cpu =1.00 

22 pr_cpu_arithlog = 0.25 

23 pr_cpu_immediate = 0.25 

24 pr_cpu_shexdet = 0.00 

25 pr_cpu_mem = 0.50 
26 

27 
28 
29 

30 pr_cpu_arithlog = 0.25 
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1 pr_add = 0.04 

2 pr_addl = 0.04 
3 

4 
5 

6 pr_sub = 0.04 

7 pr_subb = 0.04 
8 

9 pr_cpu_immediate = 0.25 

10 prjdo =0.15 
11 
12 
13 

14 pr_subi =0.10 

15 

16 

17 

18 With the above-assigned probabilities, the RCG 20 will never select a floating 

1 9 point (fp) instruction (i.e., prfp = 0.00). Note that the cimiulative probability of the first 

20 segment is 1 .00. That is, pr_cpu and pr_fp sum to 1 .00. This cumulative probability rule 

21 is maintained throughout the probability file. 

22 By assigning higher probability values to some instructions and lower probability 

23 values to other instructions, the test designer can ensure that all binary instructions are 

24 eventually tested. 

25 The RCG 20 begins with a seed value and then determines a pseudo random 

26 probability value. For example, a seed value of 10 may produce a probability sequence 

27 of 0.34, 0.8, .... Using the cumulative probability in the above example, a random 

28 probability of 0.34 would lead the RCG 20 to generate a cpu_immediate instruction, and 

29 a random probability of 0. 8 would lead the RCG 20 to generate a pr cpu immediate subi 

30 instruction. 
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1 Figure 2 is a flowchart of a process that may be used by the framework 1 0 of 

2 Figure 1 . The process includes a main process and sub-processes A and B. Sub-process 

3 A operates on the native platform 1 1 and the X execution engine 30. Sub-process B may 

4 operate in one of at least two ways. First, sub-process B may operate on the binary 

5 emulator 45, which may in turn operate on the target platform 12. Second, sub-process 

6 B may operate on the binary emulator 45, but the binary emulator 45 may operate on a 

7 target platform simulator. The target platform simulator may then operate on the native 

8 platform 11. 

9 The main process begins at 100. In code generate block 110, the RCG 20 

10 generates pseudo-random native machine code and generates a random initial machine 

1 1 state. The native machine code and the initial machine state are provided to the native 

1 2 platform 1 1 , and the native platform 1 1 is initialized using the initial machine state, block 

13 120. The same native machine code and initial machine state are also provided to the 

14 target platform 12, and the target platform 12 is initialized, block 130. The native 

15 machine code and initial machine state may be provided to the target platform 12 using 

16 TCP/IP protocols, for example. 

17 In block 140, the X execution engine 30 executes the native machine code. 

1 8 Concurrently, the binary emulator 45 emulates the entire Y execution engine 40, which 

19 intemally executes the native machine code, block 150. Next, a final machine state SI 

20 is collected in the native platform 1 1 , block 1 60, and a final machine state S2 is collected 

21 in the target platform 12, block 1 70. 

22 In block 180, the target platform 12 provides the final machine state S2 to the 

23 verification engine 50, using TCP/IP protocols, for example. The verification engine 50 

24 compares the final machine states SI and S2. If the final machine states SI and S2 are 

25 equal, the process moves to block 220 and ends. If the final machine states S 1 and S2 are 

26 not equal, the process moves to block 210, and information related to the emulation 

27 failure (i.e., the initial and final states, and the instruction sequence) are written to a file. 

28 The process then moves to block 220 and ends. 

29 The test designer then need only refer to the file to determine the source of the 

30 emulation failure, and to reproduce such emulation failure. 
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1 The terms and descriptions used herein are set forth by way of illustration only 

2 and are not meant as limitations. Those skilled in the art will recognize that many 

3 variations are possible within the spirit and scope of the invention as defined in the 

4 following claims, and there equivalents, in which all terms are to be understood in their 

5 broadest possible sense unless otherwise indicated. 
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